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Abstract 

Recent  world  events  have  demonstrated  that  future  conflicts  will  not  necessarily  be  between 
nation  states.  Transnational  threats  to  U.S.  national  security  include  drug  cartels  and  inter¬ 
national  criminal  and  terrorist  organizations.  In  response,  the  U.S.  Joint  Forces  Command  is 
leading  a  transition  toward  a  new  approach  to  warfare.  A  cornerstone  of  this  new  approach  is 
the  concept  of  Rapid  Decisive  Operations,  which  integrate  knowledge,  command  and  control, 
and  Effects-Based  Operations  to  achieve  the  desired  strategic  outcome  or  “effect”  on  the 
enemy  through  the  synergistic  application  of  the  full  range  of  military  and  nonmilitary 
capabilities  at  all  levels  of  conflict.  In  preparing  for  and  conducting  a  RDO,  the  military  acts 
in  concert  with  and  leverages  the  other  instruments  of  national  power  to  understand  and 
reduce  the  adversary’s  critical  capabilities  and  coherence.  Focusing  on  effects,  rather  than 
attrition,  enables  a  highly  coordinated  level  of  inter-service,  interagency,  and  international 
cooperation.  An  experiment  was  conducted  to  examine  aspects  of  EBO,  and  to  specifically 
assess  and  refine  the  effects-based  planning  process.  This  paper  describes  the  results  of  a 
survey  on  the  effects-based  planning  and  assessment  process,  and  is  a  companion  paper  to 
two  other  papers  that  discuss  results  of  the  other  two  surveys  that  were  administered. 

1.  Introduction 

The  world  and  operating  environment  for  the  U.S.  military  are  changing.  Recent  world  events  have 
demonstrated  that  future  conflicts  will  not  necessarily  be  between  nation  states.  Transnational  threats  to 
U.S.  national  security  include  drug  cartels  and  international  criminal  and  terrorist  organizations. 
Additionally,  the  end  of  the  cold  war  has  led  to  reductions  in  U.S.  forces  and  loss  of  overseas  bases, 
while  at  the  same  time.  Congress  has  expressed  a  desire  for  a  more  rapid  resolution  of  future  conflicts. 
In  response,  the  U.S.  Joint  Eorces  Command  (JFCOM),  J9,  is  leading  a  transition  toward  a  new 
approach  to  warfare. 

A  cornerstone  of  this  new  approach  is  the  concept  of  Rapid  Decisive  Operations  (RDO),  which  integrate 
knowledge,  command  and  control,  and  Effects-based  Operations  (EBO)  to  achieve  the  desired  strategic 
outcome  or  “effect”  on  the  enemy  through  the  synergistic  and  cumulative  application  of  the  full  range  of 
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military  and  nonmilitary  capabilities  at  all  levels  of  conflict.  ’  In  preparing  for  and  conducting  a  RDO, 
the  military  acts  in  concert  with  and  leverages  the  other  instruments  of  national  power  to  understand  and 
reduce  the  adversary’s  critical  capabilities  and  coherence.  Focusing  on  effects,  rather  than  attrition, 
enables  a  highly  coordinated  level  of  inter-service,  interagency,  and  international  cooperation. 

This  new  approach  to  warfare  requires  a  highly  knowledgeable  understanding  of  the  direct,  indirect, 
intended,  and  unintended  consequences,  or  effects,  of  our  actions.  To  provide  an  example,  a  traditional 
attack  on  a  weapons  manufacturing  plant,  or  an  industrial  target,  may  be  less  effective  than  in  the  past  if 
the  equipment  that  was  destroyed  and  the  lost  production  can  be  quickly  and  easily  replaced  in  the 
international  marketplace.  Moreover,  because  the  industrial  facility  in  a  hostile  country  may  be  owned 
by  a  firm  in  a  friendly  country,  this  decision  to  attack  is  further  complicated.  The  decision  to  attack  a 
particular  target  requires  a  much  more  elaborate  understanding  and  forecasting  of  collateral,  indirect, 
and  unintended  consequences  than  was  used  in  planning  attacks  in  previous  adversarial  situations. 

The  shift  to  EBO  effects  both  planning  and  assessment.  Traditionally,  military  planners  would  focus  on 
consideration  of  factors  such  as  the  number  and  type  of  aircraft  to  be  employed,  types  and  quantity  of 
weapons  to  be  used,  and  the  number  of  sorties  to  be  flown.  Now  the  goal  is  to  emphasize  what  all  these 
weapons  and  platforms  will  accomplish,  that  is,  what  overall  outcomes  will  be  achieved.^  Similarly,  the 
effects  assessment  (EA)  process  goes  beyond  traditional  attrition-related  and  performance-related 
assessments  included  in  today’s  combat  assessment  process.  EA  must  be  a  continuous  feedback  process 
that  can  collect,  process,  exploit,  and  disseminate  information  to  the  appropriate  level  (strategic, 
operational,  or  tactical)  in  time  spans  that  allow  commanders  to  make  decisions  in  an  intensely  dynamic 
situation.  The  EA  perspective  includes  “determination  of  whether  some,  or  all  of,  the  desired  effects 
were  produced  and,  if  so,  to  what  level,  what  unintended  effects  were  produced,  their  overall  impact  on 
the  joint  effort,  and  how  the  tactical  actions  taken  either  contributed  or  failed  to  contribute  to  obtaining 
the  desired  outcome.”  The  experiment  reported  on  in  this  paper  is  part  of  a  sequence  of  experiments 
and  events  planned  to  test  and  refine  new  concepts  being  developed  at  JECOM,  J9. 

2.  Effects-Based  Operations  Planning  Process 

The  effects-based  operations  (EBO)  process  is  conceptualized  as  a  continuous  and  iterative  planning  and 
execution  cycle.  This  cycle  comprises  the  following  five  phases.  Developing  a  comprehensive  know¬ 
ledge,  or  insight,  into  the  nature  of  the  adversary,  the  environment,  and  our  own  capabilities;  this  is 
accomplished  by  conducting  an  operational  net  assessment  (this  process  is  described  below).  The  second 
phase  entails  articulating  the  desired  effects  necessary  to  break  the  adversary’s  cohesion  and  to  cause  the 
adversary  to  change  his  behavior — these  effects  are  articulated  via  an  Effects  Tasking  Order  (ETO).  The 
third  phase  involves  determining  and  applying  those  elements  of  national  power,  i.e.,  diplomatic, 
informational,  military,  and  economic,  that  will  be  most  effective  in  achieving  the  desired  effects,  and 
assessing  the  application  of  capabilities  in  terms  of  the  desired  effects.  The  forth  phase  includes 
conducting  an  integrated  and  continuous  assessment  process  to  measure  and  assess  the  impact  of  the 
effects  created.  The  fifth  phase  involves  making  decisions  regarding  ways  the  commander  can  adapt  and 
adjust  the  current  course  of  action  to  more  effectively  reach  the  desired  end  state. 

Characteristics  that  distinguish  effects-based  planning  from  traditional  objectives-based  planning 
include  development  of  a  broader  and  deeper  insight  into  the  adversary  through  the  fusion  of 
information  from  a  broad  spectrum  of  sources,  including  national  and  international,  government  and 
non-govemmental.  Insight  into  the  adversary  is  developed  through  conducting  a  complex  “system  of 
systems  analysis”  (SoSA)  of  the  adversary.  Planners  conduct  this  systems  analysis  collaboratively  with  a 


networked  and  distributed  team  of  experts  (cultural,  behavioral,  technical,  economic,  political,  and 
military)  and  centers  of  excellence.  EBO  planning  goes  a  step  further  than  consideration  of  which 
actions  will  produce  an  objective  (i.e.,  the  traditional  approach)  to  consider  the  full  range  of  potential 
results  of  our  actions  (including  direct  and  indirect  effects,  desired  and  undesired  effects). 

3.  Operational  Net  Assessment 

The  challenge  inherent  in  the  EBO  process  is  to  accurately  identify  the  causal  linkages  that  determine 
whether  or  not  the  action  taken  will  achieve  the  desired  effect.  This  challenge  is  addressed  through 
the  Operational  Net  Assessment  (ONA).  The  ONA  is  a  product  of  collaboration  among  analysts  at 
strategic,  operational,  and  tactical  levels.  It  provides  a  common  knowledge  base  available  to  all 
customers  from  the  national  strategic  level  to  the  tactical  level.  Contributions  of  potential  customers 
impact  each  element  of  the  ONA,  since  the  ONA  views  a  potential  adversary  as  an  interdependent 
system  of  systems,  all  of  which  contribute  to  some  degree  toward  his  societal  coherence,  will,  and 
capability  to  pursue  a  course  of  action  inimical  to  friendly  interests.  The  ONA  goes  far  beyond 
traditional  intelligence  assessments.  It  is  an  action-oriented  process  that  provides  a  continuous  stream  of 
knowledge  from  adversary  vulnerabilities  to  effects  to  tasks.  The  ONA  process  helps  develop  situational 
awareness  and  understanding. 

4.  Concept  Experiment  on  the  Effects-Based  Planning  Process 

An  experiment  entitled  Effects  Tasking  Order-to-Actions  Eimited  Objective  Experiment  (ETO-to- 
Actions  EOE),  was  conducted  at  the  JECOM,  Joint  Experimentation  Center,  Suffolk,  VA,  3-14 
December,  2001,  to  examine  aspects  of  EBO,  and  to  specifically  assess  and  refine  the  effects-based 
planning  and  assessment  (EBP&A)  processes.  This  experiment  was  designed  and  conducted  by  a 
partnership  of  the  Naval  Postgraduate  School,  JECOM,  J9,  and  the  Navy  Warfare  Development 
Command.  The  previous  experiments  and  events  focused  primarily  on  the  Joint  Eorce  Headquarters 
(JEHQ)  level  of  command,  with  the  majority  of  effort  directed  at  refining  the  process  down  to  producing 
an  ETO  (which  effectively  issues  effects-based  orders  to  the  subordinate  functional  components).  The 
ETO-to-Actions  EOE  focused  within  and  below  the  JEHQ  to  examine  the  required  coordination  and 
collaboration  processes  (both  vertically  and  horizontally)  between  the  JEHQ  and  the  functional 
components  HQs,  which  is  needed  to  collaboratively  develop  the  ETO  and  translate  the  effects  directed 
in  it  into  tactical  actions  on  the  battlefield. 

The  experimental  participants  included  representatives  from  all  services,  organized  into  a  Joint  Task 
Eorce  HQ  staff  and  functional  component  HQ  staffs  (air,  maritime,  land,  special  operations  force,  and 
political).  Thirty-nine  percent  were  active  duty,  forty-two  percent  were  retired  military  personnel,  and 
the  rest  were  civilian  concept  developers  at  JECOM.  Three  days  of  training  on  the  new  processes  and 
collaboration  tools  were  provided  to  familiarize  participants  with  the  new  concepts.  Eive  days  of  game 
execution  consisted  of  the  collaborative  planning  between  the  Joint  Task  Eorce  staff  and  the  subordinate 
functional  component  staffs,  culminating  in  the  JTEC  issuing  an  ETO  to  the  Joint  Eorce. 

5.  Survey  Administration 

Three  surveys  were  administered  during  the  experiment  to  gather  data  on  the  effectiveness  of  (1)  the 
collaborative  tools  and  the  training  provided  to  the  participants  for  this  experiment,  (2)  knowledge 
management  and  collaboration  as  critical  aspects  of  effects-based  planning,  and  (3)  the  effects-based 
planning  and  assessment  process.  This  paper  describes  the  results  of  the  survey  on  the  effects-based 
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planning  and  assessment  process,  and  is  a  companion  paper  to  two  other  papers  that  discuss  results  of 
the  other  two  surveys."^’  ^ 

Forty-three  items  were  presented  as  statements  about  the  effects-based  planning  and  assessment  process. 
This  paper  will  present  results  of  a  sub-set  of  these  items  due  to  space  limitations.  (These  items  are 
listed  in  Table  1.)  Participants  were  asked  to  rate  the  extent  to  which  they  agreed  with  each  statement  by 
using  a  five -point  Lickert  scale  that  ranged  from  “strongly  disagree”  to  “strongly  agree”  (where  1  = 
strongly  disagree,  2  =  somewhat  disagree,  3  =  neither  disagree  nor  agree,  4  =  somewhat  agree,  and  5  = 
strongly  agree).  Nine  of  the  items  were  open-ended  questions  and  asked  participants  to  indicate  ways  in 
which  various  aspects  of  the  process  could  be  improved.  The  survey  was  administered  to  all  99 
experimental  participants;  88  completed  surveys  were  returned  and  analyzed.  Comments  on  the  survey 
were  reviewed  and  are  summarized  in  this  paper. 

6.  Results 

Table  1  presents  the  survey  items,  the  mean  ratings  given  by  the  participants,  and  the  standard  deviation 
for  each  rating.  Overall,  the  participants'  ratings  for  the  survey  items  varied  considerably  depending  on 
the  survey  question.  The  goal  for  this  experiment  was  to  help  define  and  further  refine  aspects  of  the 
EBP&A  process.  In  most  cases,  items  that  received  a  low  mean  rating  reflect  the  fact  that  parts  of  the 
EBP&A  process  need  additional  development.  Each  of  the  items  will  be  discussed  in  the  order  in  which 
they  are  listed  in  the  table. 

Table  1.  Effects-Based  Planning  and  Assessment  Survey  Questions  and  Results. 


Item 

Num. 

Question 

Mean 

Std. 

Dev. 

1 

The  procedures  for  effects-based  planning  were  clear  and  complete. 

2.9 

1.1 

2 

JTF/Component  responsibilities  for  EBP&A  were  clearly  defined. 

3.0 

1.0 

3 

I  experienced  uncertainty  regarding  desired  effects  passed  down  from  more 
senior  commanders  when  developing  plans  at  my  level. 

2.8 

1.1 

4 

The  EBP&A  process  was  more  challenging  than  traditional  objectives-based 
planning. 

3.6 

1.2 

5 

During  effects-based  planning,  the  process  of  updating  the  ONA  for  a  new 
potential  effect  (nodes,  links,  tasks)  was  timely. 

2.6 

1.1 

6 

What  data/information/assessments  should  the  components  provide  in  the  ETO 
to  better  enable  effects-based  planning  and  assessment  (EBP&A)? 

❖ 

❖ 

7 

The  effects-based  planning  process  facilitates  the  development  of  more  complete/ 
robust  courses  of  action. 

3.2 

1.1 

8 

The  development  of  only  one  COA  provided  sufficiently  in-depth  planning  to 
properly  develop  branches  and  sequels  in  anticipation  of  adversary  actions, 
reactions,  and  counteractions. 

2.8 

1.1 

9 

Daily  staff  briefings  contained  aspects  of  effects-based  operations. 

3.8 

.98 

10 

JTE  planners  were  able  to  develop  Measures  of  Effectiveness  (MOEs)  for  the 
desired  effects  that  were  meaningful  and  observable. 

3.0 

.93 

11 

Use  of  the  Prioritized  Effects  List  (PEL)  supports  JTE  level  decisionmaking. 

3.5 

.99 

12 

There  are  too  many  participants  involved  in  the  EBP  process. 

2.7 

1.1 

13 

It  is  easy  to  know  when  the  Effects-Tasking  Order  (ETO)  is  being  updated. 

2.6 

1.1 

14 

Effects-Based  Assessments  were  completed  and  posted  to  the  CROP  in  a  timely 

2.6 

.94 

5 


enough  fashion  to  support  current  and  future  planning  efforts. 

15 

CONOPs/TTPs  used  during  the  ETO-to-Actions  LOE  clearly  describe  how  the 
EBP&A  process  and  the  JISR  management  process  complement  each  other. 

2.7 

1.0 

16 

Component  participation  is  required  for  effects-based  planning. 

4.6 

.79 

17 

Component  participation  in  the  JTE  mission  analysis  process  facilitated 
component-level  and  cross-component  planning  efforts. 

4.1 

.89 

18 

Component  participation  in  the  JTE  course  of  action  development  process 
facilitated  component-level  and  cross-component  planning  efforts. 

4.2 

.86 

Notes.  *  Indicates  a  text  only  response. 


6.1  Procedures  for  EBP&A  were  clear  and  complete. 

The  assertion  that  the  procedures  for  EBP&A  were  clearly  defined  received  a  mean  rating  of  2.9,  slight¬ 
ly  below  the  midpoint.  This  equivalency  most  likely  reflects  that  the  assessment  portion  of  EBP&A  was 
not  included  in  this  EOE,  and  was  reflected  in  the  varying  degree  of  agreement  or  disagreement  with  the 
assertion  in  the  player  comments.  One  participant  who  agreed,  commented  that  effects-based  planning 
is  not  new,  the  procedures  have  been  clear  for  years  (i.e.,  they  are  taught  at  the  War  College  and 
practiced  on  warfighting  CINC  staffs  and  at  the  Joint  Warfare  Analysis  Center).  This  player  further 
stated  that  the  main  value  added  during  the  EOE  was  the  greater  emphasis  on  formal  interagency  partici¬ 
pation,  and  that  on  the  intelligence,  surveillance,  and  reconnaissance  (ISR)  side,  things  like  naval 
intelligence  support  teams  would  become  even  more  important.  Another  expressed  uncertainty  about 
what  was  new,  and  asked  whether,  “in  the  past  we  didn’t  understand  the  final  effect  was  more  important 
than  the  number  of  tons  of  ordnance  we  dropped?”  Several  players  who  disagreed  stated  that  the  basics 
are  there,  but  exactly  what  happens,  who  talks  to  who,  what  empowering  products  are  produced,  and 
posted,  where. .  .are  still  a  work  in  progress,  e.g.,  the  terminology  is  not  universally  accepted  yet. 

Several  problems  were  noted  with  the  process  or  its  implementation.  One  player  stated  that  ONA  use 
has  not  been  linked  very  well  to  tactical  level  planning,  and  noted  that  the  joint  force  maritime 
component  commander  (JEMCC)  had  aggressively  used  the  ONA  to  do  tactical  level  planning,  but  this 
had  been  an  “a  la  carte”  approach  to  the  analysis.  This  player  further  commented  that  the  ONA  analysts 
need  an  online  collaboration  forum  to  clarify  the  connection  between  the  system  of  systems  analysis 
(SoSA)  and  tactical  actions.  Another  stated  that  the  concepts  are  not  difficult  themselves,  but  they 
compete  with  existing  ideas  of  how  planning,  production  and  communication  of  orders  should  be 
conducted.  Yet  another  participant  noted  that  it  is  still  not  clear  how  elements  of  effects-based  planning 
dovetail  with  existing  procedures.  This  player  asked  if  the  procedures  were  designed  to  replace,  or  add 
to,  the  current  process;  specifically,  for  the  targeting  process  and  the  collection  process.  Do  nodes 
replace  targets?  Is  the  target  list  built  from  data  stored  in  the  ONA? 

6.2  JTF/Component  responsibilities  for  EBP&A  were  clearly  defined. 

The  question  that  asked  whether  JTE  and  component  responsibilities  for  EBP&A  were  clearly  defined 
received  a  mean  rating  of  3.0,  accordingly,  the  comments  fell  on  both  sides  of  the  issue.  Some 
participants  felt  that  the  responsibilities  were  clearly  defined  at  some  levels  of  the  force,  but  not  at 
others.  Others  stated  that  they  were  well  defined  at  the  macro  level,  but  the  details  need  to  be  filled  in, 
such  as  the  meaning  of  these  responsibilities,  in  terms  of  planning,  execution,  and  synchronization 
functions,  is  not  defined.  Commenting  on  the  relationships  between  components,  one  participant 
observed  that  a  delineated  organizational  matrix,  which  shows  interactions  and  periodicity,  is  needed. 
Another  stated  that  there  was  confusion  as  to  the  degree  of  authority  that  the  supported  component 


commander  has,  questioning  whether  he  actually  develops  the  entire  plan  himself  and  then  turns  it  over 
to  the  JTF.  Yet  another  stated  that  while  many  responsibilities  were  well  defined,  the  requirement  for 
synchronization  among  services,  vice  the  JTF  leading  the  effort  was  not. 

Another  player  emphasized  that  this  was  an  experiment,  and  the  objective  was  to  work  through  the 
issues  to  determine  what  worked  well  and  what  did  not.  Moreover,  some  degree  of  "fog"  is  inherent  to 
this  process.  One  player  stated  that  the  JFMCC,  in  an  effort  to  incorporate  lessons  learned,  critiqued  and 
modified  a  process  model  that  had  been  developed  prior  to  the  LOE.  This  was  followed  by  a  suggestion 
that  the  JTFC  staff  similarly  modify  its  process  model  to  show  how  interactions  between  components 
and  the  JTF,  and  between  the  service  components,  are  expected  to  work.  Effects  assess-ment  was  noted 
as  a  weakness.  One  player  stated  that  responsibilities  had  been  refined  considerably  over  the  course  of 
the  EOE,  but  assessment  is  needed.  Another  commented  that  the  assessment  part  was  in  particular  a 
problem,  further  stating  that  the  ability  to  assess  a  smoking  hole  is  much  easier  than  assessing  an  'effect', 
and  noted  that  the  methods  and  tools  to  conduct  effects-based  operations  will  require  much  more 
thought.  Reflecting  on  EBP&A,  one  participant  commented  that  stating  an  effect  (to  be  achieved)  is 
easy,  determining  how  to  achieve  that  effect,  is,  and  has  always  been,  the  challenge. 

Some  players  thought  the  actual  processes  for  developing  an  effects-based  campaign,  which  in  turn  is 
translated  into  an  ETO  for  conducting  synchronized  simultaneous  joint  tactical  actions,  with  supporting 
collection  mechanisms  to  assess  effects,  are  not  well  defined  or  articulated.  Therefore,  only  the  macro 
component  responsibilities  (e.g.,  plan  and  execute  joint  tactical  actions  (JTAs),  or  supporting  actions,  for 
other  component  JTAs  are  well  defined).  What  these  responsibilities  mean,  in  terms  of  planning,  and 
execution  of  synchronized  functions,  is  not  defined. 

6.3  I  experienced  uncertainty  regarding  desired  effects  passed  down  from  more  senior 
commanders  when  developing  plans  at  my  level. 

The  question  that  asked  if  participants  experienced  uncertainty  regarding  desired  effects  passed  down 
from  more  senior  commanders  when  developing  plans  at  their  level  received  a  mean  rating  of  2.7. 
Comments  indicate  that  the  desired  effects  were  for  the  most  part  understood,  but  how  to  translate  them 
into  plans  at  the  lower  level  was  less  clear.  Comments  from  players  who  had  experienced  uncertainty 
included  that  the  expressed  effects  desired  were  too  broad,  and  that  a  geographic  representation  of  the 
CJTE  plan  with  greater  specificity  regarding  desired  effects  would  be  very  helpful.  Those  who  did  not 
experience  uncertainty  regarding  desired  effects  thought  that  the  desired  effects  were  clear  and  that  the 
effects  called  for  could  be  converted  into  plans.  Others  indicated  that  the  difficulty  was  translating  the 
high-level  effects  into  actionable  things  to  do.  Eor  example,  the  JTE  Commander’s  guidance  was  very 
clear,  it  was  everyone  else’s  interpretation  of  this  guidance  that  created  uncertainty;  others  indicated  that 
the  desired  effects  were  clear,  but  prioritization  regarding  desired  effects  was  not. 

Problems  identified  as  contributing  to  uncertainty  included  that  the  totality  of  how  to  achieve  effects  was 
still  not  clearly  understood.  It  was  not  clear,  for  example,  whether  the  JTE  staff  could  resolve  different 
approaches  to  achieving  effects  that  the  components  might  develop  since  no  functional  process  or  tool 
for  force  deconfliction  exists.  A  related  view  was  that  broad  effects  at  the  CINC  level  should  be  paired 
down  by  the  components  to  develop  effects  that  are  achievable  by  their  commands  (as  is  done  with 
commander’s  intent  today).  An  example  of  the  latter  was  that  a  traditional  item  from  commander's 
intent,  e.g.,  "ensure  freedom  of  navigation"  would  change  to  "confine  enemy  fleet  to  mainland  ports." 

6.4  The  EBP&A  process  was  more  challenging  than  traditional  ohjectives-hased  planning. 
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The  question  that  asked  whether  the  EBP&A  process  was  more  challenging  than  traditional  objectives- 
based  planning  received  a  mean  rating  of  3.6.  Players  who  thought  that  the  EBP&A  process  is  more 
difficult  thought  it  was  for  the  following  reasons:  EBP&A  requires  extensive  collaboration  to  be 
successful;  it  is  new,  e.g.,  there  is  a  good  deal  of  new  nomenclature  (in  addition  to  the  fact  that  they  were 
simultaneously  working  on  refining  the  concepts  in  addition  to  developing  the  effects-based  planning 
products);  it  is  not  as  straightforward  as  simply  targeting  known  enemies;  and  it  represents  a  change  in 
the  way  one  thinks.  A  cogent  comment  was  that  while  EBP&A  is  new,  even  after  it  matures  it  will  still 
be  more  difficult  than  an  objective-based  approach  to  planning,  e.g.,  “take  that  hill.”  Despite  the 
inherent  challenges  associated  with  EBP&A,  others  felt  that  the  benefits  will  outweigh  the  extra  work 
entailed;  the  potential  payoffs  in  precision  and  efficiency  are  enormous;  however,  the  assessment  part  of 
the  process  will  be  much  more  difficult  if  EBP&A  is  going  to  be  effective. 

Players  who  did  not  think  the  EBP&A  process  was  more  challenging  thought  that  there  was  no  signifi¬ 
cant  difference  in  the  two  processes:  "There  is  a  misperception  that  objectives-based  planning  discounts 
effects,  and  that  effects-based  planning  discounts  operational  and  strategic  objectives."  A  similar 
perspective  was  that  the  basics  are  no  more  challenging:  the  challenge  is  understanding  the  concept  and 
relationships.  Others  felt  that  using  the  tools  and  learning  to  navigate  around  inside  the  tools  was 
difficult,  but  not  the  entire  process;  planning  and  execution  in  a  joint  operational  environment  has 
always  been  challenging;  and  that  it  is  not  more  challenging  than  doing  traditional  objectives-based 
planning  correctly.  Problems  identified  included  the  integration  of  collaborative  planning;  balancing 
time  vs.  producing  the  planning  products;  and  continuously  thinking  and  expressing  oneself  in  terms  of 
"effects."  Collaborative  planning  was  difficult  because  doing  things  collaboratively  is  new,  and  in  many 
cases,  the  players  were  not  familiar  with  the  other  participants,  so  they  were  not  sure  who  to  collaborate 
with  or  how  the  collaboration  process  was  to  be  conducted. 

6.5  During  effects-based  planning,  the  process  of  updating  the  ONA  for  a  new  potential  effect 
(node,  links,  tasks)  was  timely. 

The  question  asserting  that  the  process  of  updating  the  ONA  was  timely  received  a  mean  rating  of  2.6, 
the  second  lowest  rating  of  any  item.  Two  participants,  who  observed  that  when  the  ONA  was  updated 
that  it  was  timely,  also  noted  that  there  was  not  much  action  to  keep  up  with  and  cautioned  not  to 
interpret  this  as  an  indication  that  the  process  of  updating  the  ONA  will  work  well  in  a  full-scale  event. 
One  added  that  in  a  live  operation  they  would  be  busy  fighting  the  war,  and  updating  the  ONA  would 
probably  not  be  the  highest  priority.  Another,  an  experienced  operations  planner  who  added  one  effect  to 
the  ONA,  was  able  to  accomplish  the  entire  process  within  30  minutes,  based  on  the  use  of  pre¬ 
identified  nodes.  This  participant  thought  that  the  process  would  have  taken  more  time  to  accomplish,  if 
the  resources  to  be  used  were  used  collaboratively.  [This  is  interpreted  to  mean  there  is  an  "overhead" 
cost  associated  with  using  asynchronous  collaboration;  this  may  be  due  to  working  in  a  multi-tasking 
environment  to  conduct  planning.]  One  participant  stated  that  as  the  process  matures,  and  all  operators 
and  planners  become  familiar  with  the  ONA,  the  process  of  updating  the  ONA  will  be  able  to  be 
accomplished  in  less  time.  Another  thought  that  once  the  process  of  using  the  ONA  tool  was  learned,  it 
was  timely. 

In  contrast,  a  participant  stated  that  the  process  of  updating  the  ONA  was  not  really  tested  or  exercised. 
Another  player  indicated  that  he  did  not  see  any  updates  while  using  the  ONA  to  develop  the  prioritized 
effects  list  (PEE);  and  another  felt  that  the  new  process  that  was  used,  and  the  resultant  changes  that 
were  produced,  were  not  always  timely.  With  respect  to  the  components,  one  player  did  not  notice  any 
updates  in  the  ONA  at  the  component  level;  another  noted  that  there  was  insufficient  familiarity  with  the 
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ONA  to  allow  for  timely  updating,  but  there  was  an  increasing  attempt  by  the  components  to  do  so.  A 
player  who  thought  the  process  of  updating  the  ONA  was  not  timely,  questioned  who  was  performing 
this  task,  and  when,  and  also  inquired  about  the  validation  process,  such  as,  who  is  the  central 
clearinghouse  for  changes?  One  participant  noted  that  the  R  [rapid]  in  RDO  appears  to  be  difficult  to 
obtain  using  the  ONA  tool  to  craft  new  effects  during  crisis  action  planning,  but  expected  there  to  be  no 
impact  on  deliberate  planning.  A  player  who  did  not  use  the  ONA  tool,  because  he  did  not  find  it  useful, 
did  not  see  how  in  real  life  it  could  possibly  be  updated  correctly. 

Several  players  noted  the  need  for  a  more  clearly  defined  process.  One  noted  that  there  are  too  many 
steps  to  be  accomplished  in  the  ONA — it’s  not  yet  up  to  Napoleon’s  "Corporal  standards"  (i.e.,  simple 
and  straightforward  enough  to  be  easily  understood).  In  partial  contrast,  one  participant  commented  that 
the  mechanics  are  very  straightforward  for  the  SoSA  and  for  the  JTFHQ,  but  the  process  of  identifying 
the  need  for  a  new  effect  to  be  developed,  and  determination  of  who  would  have  the  responsibility  for 
executing  it,  were  more  difficult.  A  similar  perspective  was  that  technically,  the  process  of  updating  the 
ONA  was  easy,  and  it  could  be  accomplished  in  a  timely  way,  but  convincing  people  to  do  this  (i.e.,  use 
these  new  tools  and  processes)  was  difficult.  Several  players  stated  that  they  were  not  familiar  with  the 
process  for  updating  the  ONA,  and  one  was  not  aware  that  this  was  possible.  One  player  noted  that 
updating  the  ONA  will  require  continuous  intelligence  assessment  updates,  with  alerts  to  the  appropriate 
planners,  and  the  lack  of  these  updates  significantly  affected  the  value  of  the  LOE. 

6.6  What  data/information/assessments  should  the  components  provide  in  the  ETO  to  better 
enable  Effects-Based  Planning  and  Assessment? 

The  question  that  asked  what  data/information/assessments  should  the  components  provide  in  the  ETO 
to  better  enable  EBP&A  received  the  largest  number  of  comments.  One  participant  stated  that  the 
required  inputs  from  the  components  depended  on  where  you  were  in  the  planning  to  execution 
continuum.  Eor  campaign  planning  and  ETO  development,  the  components  need  to  provide  options  and 
feasibility  assessments  from  a  component  level.  Eor  developing  actions  and  effects,  at  the  point  of  joint 
tactical  action  planning  and  execution,  components  need  to  provide  completed,  synchronized  plans.  Eor 
the  assessment  process,  components  will  contribute  resources  to  perform  assessment,  and  integrate 
assessment  requirements  into  their  other  actions.  Another  noted  that  for  information  operations  (10), 
there  will  need  to  be  a  significant  amount  of  detail  provided  because  unlike  other  warfare  domains,  the 
10  plan  directs  specific  actions  to  components,  versus  desired  effects. 

6.6.1.  Feedback  on  actions  and  effects.  One  player  stated  that  the  components  should  provide  a  “reality 
check,”  and  subsequent  feedback,  on  the  planned  military  actions  using  the  Diplomatic,  Information, 
Military,  Economic  (DIME)  model  (which  is  based  on  using  a  synergistic,  comprehensive  approach,  that 
encompasses  these  four  elements,  to  achieve  the  desired  effects).  Another  agreed,  stating  that  the 
components  should  provide  their  assessment  of  the  effects  they  can  support,  even  if  the  actions  are  not 
things  they  have  traditionally  done;  others  felt  that  the  components  should  actually  develop  the  actions 
in  the  plan.  One  player  stated  that  the  designated  component  mission  commander  should  develop  the 
course  of  action  (CO A),  while  another  felt  that  all  the  components  should  be  involved  in  development  of 
the  COA.  Others  indicated  that  the  components  should  identify  some  enemy  nodes  to  be  attacked  and 
provide  component  actions  and  tasks,  that  would  be  used  to  do  so,  and  yet  another  that  the  components 
should  convert  the  effects  in  the  ETO  to  missions  (i.e.,  who,  what,  where,  when,  why).  Discussing  the 
maritime  component  in  particular,  a  player  stated  that  the  maritime  component  should  provide  detailed 
options  for  achieving  the  effects  that  pertain  to  maritime  warfare.  The  example  was  given  that  in 
antisubmarine  warfare,  there  are  several  ways  to  achieve  dominance  beyond  those  that  are  kinetic  based. 
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6.6.2.  Measures  of  Effectiveness  and  Performance.  More  than  a  dozen  participants  stated  that  they 
should  provide  measures  of  performance  (MOP)  and/or  measures  of  effectiveness  (MOE)  (to  assess  own 
force  performance  on  assigned  actions,  and  the  extent  to  which  desired  effects  were  achieved, 
respectively).  Of  these,  approximately  half  focused  on  the  planning  phase,  providing  the  measures  to  be 
used.  Defining  success  (i.e.,  developing  MOE  and  MOP)  is  the  most  challenging  aspect  of  the  EBO 
process,  as  it  combines  the  art  and  science  of  war  at  the  operational  level.  The  other  half  focused  on 
providing  values  for  the  measures  during  execution.  A  player  discussed  providing  feedback,  as  input  to 
the  ONA,  and  another  stated  that  there  should  be  an  intelligence-like  effects  assessment  team  and 
process  at  the  CITE  HQs,  and  within  the  components,  whose  sole  purpose  is  updating  and  maintaining 
the  ONA  and  determining  effectiveness.  Another  specifically  stated  that  component-provided  MOP 
should  not  be  in  the  ETO;  they  should  just  provide  feedback  to  the  CITE  on  progress  of  the  ETO  as  it  is 
executed.  It  was  also  pointed  out  that  it  is  the  assessment  that  will  dictate  whether  we  are  in  need  of 
planning  a  new  ETO.  On  the  other  side  of  the  assessment  coin,  one  participant  stated  that  the 
components  should  provide  feedback  on  the  effects  of  the  enemy  on  our  forces. 

6.6.3.  Joint  Intelligence  Preparation  of  the  Battlespace.  Participants  discussed  the  (Joint)  Intelligence 
Preparation  of  the  Battlespace  (JIPB/IPB)  in  response  to  this  item.  One  noted  that  processes  are  already 
in  place  to  push  combat  assessment  and  IPB  products  up  the  chain  of  command,  and  that  these  are  the 
data  needed  for  EBO.  Another  stated  that  the  components  should  provide  their  JIPB  and  updates,  noting 
anything  that  is  different  from  what  is  in  the  ONA.  He  also  noted  the  need  to  update  the  who,  how, 
why,  and  contact  information  for  the  person  doing  the  updating.  Yet  another  stated  that  the  components 
should  provide  the  IPB,  and  noted  that  it  was  something  they  didn’t  have  (in  the  EOE).  Also  in  the 
intelligence  domain,  one  player  stated  that  the  components  should  provide  human  intelligence,  and 
another  felt  that  they  should  provide  a  draft  intelligence,  surveillance  and  reconnaissance  (ISR) 
collection  plan  for  publication  in  the  ETO. 

6.6.4.  Force  capabilities.  Several  players  discussed  the  need  for  the  components  to  provide  information 
on,  and  recommendations  for,  employment  of  their  assets.  One  stated  that  the  components  should 
provide  the  asset  allocation  and  apportionment  for  the  ETO.  Another  participant  stated  that  the 
components  should  provide  information  on  current  and  future  resources,  capabilities  and  ISR  assets 
available;  another  stated  that  they  should  provide  situational  awareness  of  force  and  resource 
capabilities.  One  player  noted  that  the  components  bring  more  fidelity  to  the  actions  planned  and 
resources  allocated,  another  stated  that  they  should  provide  specific  service  capabilities,  yet  another 
commented  that  they  should  provide  real-time  status  of  force  capabilities  and  risk  analysis,  and  still 
another  that  they  should  provide  logistical  needs. 

6.6.5.  Resolve  conflicting  tasking.  Participants  also  discussed  the  role  of  the  components  in  planning 
for  the  integration  and  deconfliction  of  forces  and  actions.  One  participant  stated  that  the  components 
should  provide  information  on  deconfliction  and  coordination  in  attacking  cross-component  nodes,  two 
others  noted  that  the  components  should  be  participants  in  the  prioritized  effects  list  (PEE)  and 
synchronization  matrix  development,  and  yet  another  felt  that  they  should  help  resolve  conflicting 
tasking.  Another  player  stated  that  the  components  should  provide  the  attack  guidance  matrix,  high 
payoff  target  list,  and  the  commander’s  concept  for  fire  support/supporting  arms.  Yet  another 
summarized  his  thoughts  as,  "the  low  point  of  the  performance  curve  is  in  the  assessment  and  feedback 
area:  How  can  we  better  depict  what  has  been  done  and  by  whom?" 

6.6.6.  Input  for  ONA.  Additional  collaboration  between  the  components,  JEHQ  planners,  and  the  SoSA 
analysts  will  be  required  to  determine  what  additional  information  is  needed,  beyond  what  was  initially 
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included  in  the  ONA  database.  A  process,  and  the  associated  procedures,  to  determine  what  additional 
information  is  required,  needs  to  be  developed  and  promulgated.  Yet  another  player  stated  that  most  of 
this  should  be  done  during  collaboration,  prior  to  the  ETO  development,  since  the  ETO  is  the  product  of 
planning.  Another  player  stated  that  during  peacetime,  as  the  standing  joint  command  and  control 
element  and  the  SoSA  analysts  collaborate  to  develop  the  ONA,  the  components  should  be  building 
target  folders.  Although  these  target  folders  will  not  be  complete,  but  by  doing  this,  the  components  will 
at  least  have  a  working  knowledge  of  the  topics  and  issues.  When  developing  the  ETO,  the  components 
should  then  be  more  efficient,  as  much  of  the  information  will  already  be  validated  and  agreed  upon. 

The  ETO  is  an  order  issued  by  the  operational  headquarters  to  the  components,  and  the  components  may 
provide  input  to  the  collaboratively  developed  ETO,  or  develop  annexes  to  support  the  base  ETO,  for 
functional  areas.  Another  stated  that  the  components  do  not  provide  any  data  in  the  ETO.  Yet  another 
disagreed  with  the  ETO  concept  completely. 

6.7  The  effects-based  planning  process  facilitates  the  development  of  more  complete/robust 
courses  of  action  (COA). 

6.7.1.  Collaboration  provides  the  synergy.  The  assertion  that  the  EBP  process  facilitates  the  develop¬ 
ment  of  more  complete  or  robust  courses  of  action  received  a  mean  score  of  3.2,  slightly  above  the 
midpoint.  A  player  who  agreed,  stated  that  effects-based  planning,  when  combined  with  input  from  the 
wide  array  of  subject  matter  experts  involved,  worked  well  to  help  the  players  rapidly  develop  a 
common  understanding  of  the  best  COA.  Several  others  also  agreed,  but  with  caveats.  One  participant 
stated  that  the  EBP  process  facilitates  development  of  more  complete/robust  courses  of  action  only  when 
there  is  a  robust  war-gaming  process  and  a  real  advocate  for  RED  involved.  Another  observed  that 
including  the  ONA  and  Blue-Red  Cell  analysis  were  big  improvements  in  effects-based  planning; 
another  stated  that  it  is  only  true  if  DIME  actions  are  conducted  in  a  synchronized  way. 

6.7.2.  Collaboration  is  essential.  Another  player  thought  that  the  EBP  process  facilitates  the  develop¬ 
ment  of  more  complete/robust  courses  of  action  if  planners  and  operators  focus  on  the  effect,  and  not 
default  to  using  the  closest  traditional  action  that  seems  to  support  what  is  desired.  Several  others  felt 
that  this  statement  about  the  COA  was  true,  but  due  in  large  part  to  collaboration:  "The  EBP  process 
will  not  facilitate  development  of  more  complete  or  robust  courses  of  action  without  synchronization 
and  visibility  across  the  components."  The  collaboration  entailed  in  building  the  COA,  and  the  review 
process,  make  the  whole  process  more  complete  and  timely.  Others  thought  collaboration  makes  the 
COA  development  process  work  better;  the  collaborative  processes,  with  all  the  players  who  are 
available,  really  provides  the  ability  to  facilitate  development  of  more  complete  or  robust  courses  of 
action. 

6.7.3.  Developins  one  COA  is  limited.  Among  those  who  disagreed  with  this  statement,  one  participant 
thought  that  developing  the  COA  using  the  new  EBP&A  process  is  accomplished  in  a  way  that  is  similar 
to  the  way  planning  has  traditionally  been  done,  just  using  a  different  tool.  Another  observed  that  he 
was  not  sure  about  EBP  being  more  complete,  but  it  was  certainly  more  complex.  Another  observed 
that  he  never  saw  more  than  one  COA  generated  or  discussed,  and  felt  that  the  COA  that  was  developed 
was  less  complete  than  plans  he  had  seen  generated  in  the  same  time  frame  for  other  events.  One  player 
believed  planners  would  reach  the  same  conclusions  regarding  development  of  COAs  using  current 
planning  processes.  Another  observed  that  the  idea  that  developing  only  one  COA  will  permit  more 
time  to  be  spent  on  developing  a  more  detailed  COA  has  not  proved  true.  This  player  noted  that  since 
there  was  no  comparison  with  alternate  COAs,  a  good  argument  can  be  made  that  the  COA  developed 


does  not  have  as  much  critical  thought  as  one  would  find  in  using  the  process  that  examines  multiple 
COAs.  Another  participant  disagreed  with  the  assertion,  but  did  not  necessarily  see  this  as  a  negative. 
He  noted  that  the  process  could,  or  should,  do  the  reverse:  The  commander  can  tailor  the  package,  and  it 
therefore  may  be  less  robust  while  still  fulfilling  the  effect  (i.e.,  because  it  has  such  precise  effects). 

6.7.4.  CO  A  development  belongs  at  higher  echelons.  Three  participants  disagreed  with  the  EBP 
concept  itself.  One  stated  that  the  thinking  process  associated  with  COA  development  is  best  suited  to 
the  higher  echelons  of  command,  i.e.,  that  EBP  belongs  at  the  CINC  level  and  above.  This  player  further 
stated  that  the  desired  effects  can  then  be  translated  into  objectives  by  lower  echelons.  The  second  noted 
that  in  the  DIME  paradigm,  the  JTE  Commander  generally  will  only  have  control  over  the  M(ilitary 
element),  thus  achieving  the  integrated,  synergistic  effects  of  all  the  DIME  elements  may  present  a 
challenge. 

6.7.5.  More  comprehensive  is  not  necessarily  more  robust.  Several  participants  both  agreed  and 
disagreed  with  this  statement.  One  noted  that  EBP  includes  all  elements  of  national  power,  but  noted 
that  it  could  be  difficult  to  coordinate  these  various  elements  to  achieve  the  desired  effects.  Another 
agreed  with  the  assertion  intuitively,  but  stated  that  he  was  not  sure  that  this  had  been  demonstrated.  He 
further  stated  that  we  may  have  demonstrated  more  comprehensiveness  in  the  ETO  development,  but  not 
in  turning  that  into  actionable  combat  power.  Another  player  expressed  the  belief  that  the  more 
comprehensive  approach  used  in  EBP — to  cause  an  adversary  to  change  behavior — forces  planners  to 
adopt  a  larger  perspective.  While  this  more  comprehensive  perspective  produces  a  COA  that  is  more 
effective  at  achieving  the  objectives,  that  does  not  necessarily  mean  it  is  more  complete  or  more  robust. 
Another  player  observed  that  only  one  COA  was  developed.  His  belief  was  that  a  concept  of  operations 
(CONOP)  had  been  collaboratively  developed,  not  a  COA,  noting  that  two  or  three  COAs  are  typically 
developed,  which  enables  a  commander  to  decide  which  one  he  or  she  wants  to  select.  This  player  felt 
that  during  this  EOE  one  COA  was  developed  which  became  a  CONOP. 

6.8  Development  of  only  one  course  of  action  provided  sufficiently  in-depth  planning  to  properly 
develop  branches  and  sequels. 

The  item  asserting  that  development  of  only  one  COA  provided  sufficiently  in-depth  planning  to  pro¬ 
perly  develop  branches  and  sequels  received  a  mean  score  of  2.8.  Several  comments  focused  more  on 
the  concept  of  a  single  COA  than  on  developing  branches  and  sequels.  One  commented  that  too  much 
time  was  spent  developing  "throw  away"  COAs,  and  that  it  is  better  to  develop  one  fully  developed 
COA  that  can  be  tweaked  to  make  it  work.  Still  another  stated,  “Everything  that  I  saw  was  PowerPoint 
deep.  A  more  innovative,  aggressive  adversary  would  pose  a  real  threat  of  getting  inside  our  OODA 
(observe,  orient,  decide,  act)  loop.  One  COA  does  not  contain  sufficient  diversity  of  actions  to  defeat  a 
thinking,  proactive  adversary.”  One  participant  stated  that  we  will  always  develop  only  one  COA — the 
one  the  commander  selects,  but  that  we  should  evaluate  more  than  one  in  order  to  select  the  best  of 
alternative  approaches.  Another  commented  that  normally,  the  Commander  selects  bits  and  pieces  from 
several  courses  of  actions  to  develop  the  COA  he  wants.  Yet  another  stated  that  part  of  good  staff  work 
is  providing  commanders  with  good  choices.  One  player  stated  the  belief  that  two  COAs  should  be 
developed,  and  another  observed  that  there  weren’t  enough  unintended  effects  indentified,  so  that 
alternate  COAs  could  be  developed  based  on  those  secondary,  or  unintended,  effects. 

Others  did  discuss  branches  and  sequels,  either  explicitly  or  implicitly.  One  participant  noted  that  the 
plan  has  to  be  flexible  enough  to  accommodate  changes  in  the  enemy’s  COA.  One  player  did  think  that 
the  planning  process  used  to  develop  a  single  COA  helped  in  developing  a  set  of  options  and  plans  for 
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exploitation.  This  player  said,  "However,  there  are  lots  of  COAs  for  every  major  effect,  and  so  the 
branches  at  the  lower  levels  tangle  with  each  other  over  time."  Another  commented  that  this  single 
COA  was  produced  collaboratively  so  consensus  was  attained  during  its  formulation.  Still  another 
stated  that  he  agreed  in  concept  that  pursuing  one  COA  collaboratively  would  negate  the  requirement  for 
multiple  COA  development,  but  noted  that  we  still  need  personnel  and  time  to  do  branch  and  sequel 
planning,  which  was  not  done. 

6.9  Daily  staff  briefings  contained  aspects  of  effects-based  operations. 

The  assertion  that  daily  staff  briefings  contained  aspects  of  EBO  received  a  relatively  high  mean  score 
of  3.8.  One  player  stated  that  they  did  not  usually  conduct  daily  staff  briefings,  but  when  they  did, 
aspects  of  EBO  were  included  in  the  effects  assessment  and  Blue-Red  cell  briefs.  Yet  another  noted  that 
the  concept  was  imbedded  in  all  of  their  discussions  up  and  down  the  process.  Two  participants 
disagreed  with  the  concept  of  daily  staff  briefings.  One  stated;  “Hopefully  daily  staff  briefings  will  be 
history  as  a  result  of  tailored  CROPs  and  collaboration.”  The  other  commented  that  briefings  are  not 
good,  collaboration  is,  and  further  noted  that  true  collaboration  reduces  the  requirement  for  “dog  and 
pony  shows,”  but  senior  commanders  must  be  open  to  new  ways  of  doing  things.  Another  participant 
made  the  general  comment  that  EBO  do  not  absolve  the  CITE  from  accomplishing  normal  and  fully 
developed  CITE  planning  requirements,  such  as  JIPB,  daily  updates  of  intelligence,  and  commander’s 
actions,  force  disposition,  etc. 

6.10  JTF  planners  were  able  to  develop  Measures  of  Effectiveness  (MOEs)  for  the  desired 
effects  that  were  meaningful  and  observable. 

The  assertion  that  JET  planners  were  able  to  develop  measures  of  effectiveness  (MOEs)  for  the  desired 
effects  that  were  meaningful  and  observable  received  a  mean  score  of  3.0,  indicating  some  agreed  and 
some  disagreed  with  this  statement.  Those  who  agreed  noted  that  there  is  still  discussion  on  whether 
planners  should  be  limited  to  what  is  in  the  ONA,  or  whether  additional  MOEs  should  be  developed 
apart  from  the  ONA.  Another  who  agreed  noted  that  it  remains  to  be  seen  if  the  assertion  will  stand  up 
in  real  life.  Another  player  commented  that  the  process  worked,  but  they  didn’t  get  far  enough  in  the 
wargame  to  know  whether  the  MOEs  were  observable,  noting  that  this  takes  a  long  time. 

Two  participants  did  not  comment  on  the  assertion  with  respect  to  the  qualities  of  the  MOEs,  but  did 
comment  on  collaboration.  One  noted  that  the  MOEs  were  primarily  developed  by  the  effects  assess¬ 
ment  analysts  with  minimal  input  from  JTE  or  component  planners,  and  that  there  was  very  little 
successful  collaboration  in  developing  the  MOEs.  Two  other  participants  commented  on  the  process 
itself.  One  stated  that  the  process  is  not  fully  developed  yet,  commenting  that  this  area  requires  much 
more  detailed  experimentation  and  analysis  with  linkage  to  JISR  planning.  The  other  felt  that  the  process 
used  in  the  EOE  was  not  right,  commenting  that  the  majority  should  be  developed  during  ONA 
development  by  the  team  with  the  most  experience  and  time  and  not  during  execution.  He  further  noted 
that  there  should  be  a  fully  developed  nodal  analysis  of  the  country  red  (since  it  is  real  and  has  been  a 
potential  adversary  for  over  two  decades),  and  recommended  direct  linkage  to  JWAC/NSA/CIA 
databases  and  processes. 

6.11  Use  of  the  Prioritized  Effects  List  (PEL)  supports  JTF  level  decision  making. 

The  question  that  asked  if  use  of  the  PEL  supports  JTE-level  decisionmaking  received  a  mean  rating  of 
3.5,  which  indicates  that  the  majority  of  the  experimental  audience  found  the  PEL  useful  in  supporting 
command-level  decision  making.  However,  there  were  mixed  reviews  on  whether  effects  should  be 


prioritized  at  all.  Some  compared  it  to  today’s  process  of  prioritized  target  lists.  Regardless  of  this 
debate,  it  was  noted  that  the  PEL  is  a  powerful  tool  in  conveying  the  intent  of  the  commander.  The 
utility  of  the  PEL,  recognized  by  some  to  have  a  great  potential  for  the  future,  hinges  on  the  quality  of 
the  ONA.  The  responsibility  for  developing  the  processes  for  maintaining  and  updating  both  the  ONA 
and  PEL  must  be  clearly  delineated.  Some  noted  that  the  processes  for  building  the  PEL  were 
compartmentalized,  “the  PEL  seemed  to  be  running  in  parallel,  but  not  necessarily  coordinated  with 
other  planning  activities.”  It  was  also  noted  that  the  PEL  process  explored  in  this  event  did  not  include 
the  analytical  assessment  and  verification  process  associated  with  the  joint  prioritized  target  list  (JPTL). 

6.12  There  are  too  many  participants  in  the  EBP  process. 

The  question  that  asked  if  there  were  too  many  participants  in  the  EBP  process  received  a  mean  rating  of 
2.7  (indicating  people  did  not  think  there  were  too  many  participants  in  the  EBP  process).  Responses  to 
this  question  were  all  over  the  board  but  the  common  theme  that  permeated  all  the  responses  was  that 
the  number  of  participants  in  a  virtual  collaboration  session  must  include  the  absolute  minimum  that 
contains  the  necessary/key  personnel  to  deal  with  the  special  requirements  of  that  session.  The  second 
theme  that  echoed  throughout  the  responses  was  the  need  for  a  detailed  and  disciplined  process  to 
facilitate  the  session.  And  lastly,  it  was  recognized  that  collaboration  with  key  essential  planners  is 
necessary  to  develop  a  fully  fleshed  out  ETO. 

As  the  joint  force  expands  their  focus  to  include  all  the  elements  included  in  the  DIME  concept  (i.e.,  the 
synergistic  use  of  diplomatic,  informational,  military,  and  economic  approaches  to  deal  with  an 
adversary),  and  their  ability  to  effect  our  adversary’s  PMESII  (political,  military,  economic,  social, 
information,  and  infrastructure),  the  input  of  more  and  more  players  will  be  required.  It  is  difficult  to 
engage  this  number  of  people  and  not  support  some  level  of  consensus  building  which  likely  will 
lengthen,  not  shorten,  our  OODA  (observe,  orient,  decide,  act)  loop.  The  challenge  will  be  to  find  the 
right  mix  between  consensus  building  and  decisionmaking  necessary  to  provide  our  commanders  with 
the  best  information,  at  the  right  time,  to  support  critical  decisionmaking. 

6.13  It  is  easy  to  know  when  the  ETO  is  being  updated. 

The  question  that  asked  if  it  was  easy  to  know  when  the  ETO  was  being  updated  received  a  mean  score 
of  2.6,  indicating  that  participants  tended  to  disagree  with  this  statement.  The  majority  of  the  comments 
focused  on  shortfalls  in  the  area  of  knowledge  management  in  terms  of  providing  users  with  better  tools 
and  systems.  The  comments  highlighted  difficulties  in  knowing  which  ETOs  (or  any  other  documents 
for  that  matter)  were  updated.  The  tool  used  to  support  the  EBP  process  did  not  include  an  automatic 
updating  feature,  so  it  did  not  automatically  highlight  changes  to  the  various  versions  of  the  documents 
that  were  being  produced.  Many  ETOs  and  PELs  were  posted  in  the  collaborative  information 
environment  that  were  authoritative,  however,  it  was  not  clear  which  were  drafts,  outdated,  updated  or 
current.  One  specific  comment  suggested  that  a  geographic  representation  of  the  CITE  plan  with  more 
detail  included  would  be  very  helpful.  Some  comments  that  suggested  that  tracking  changes  and  being 
aware  of  the  latest  changes  were  not  problems.  This  might  suggest  that  the  tools  are  capable  but  not  all 
users  were  properly  trained  or  experienced.  Obviously  this  is  an  area  that  needs  continued  attention. 

6.14  Effects-hased  assessments  were  completed  and  posted  to  the  CROP  (common  relevant  oper¬ 
ational  picture)  in  a  timely  enough  fashion  to  support  current  and  future  planning  efforts. 

The  question  of  whether  effects-based  assessments  were  completed  and  posted  to  the  common  relevant 
operational  picture  (CROP)  in  a  timely  enough  fashion  to  support  current  and  future  planning  efforts 
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received  a  score  of  2.6.  Comments  indicated  that  the  effects  assessment  process  was  one  of  the  weakest 
areas  of  the  LOE.  A  player  recommended  that  the  area  of  effects  assessment  should  be  the  subject  of  an 
entire  LOE.  Effects  assessment  was  highlighted  in  the  quicklook  report  as  one  of  three  key  areas  that 
need  significant  refinement  prior  to  Millennium  Challenge  02. 

6.15  CONOPs/TTP  used  during  the  ETO-to-Actions  LOE  clearly  describe  how  the  EBP&A 
process  and  the  JISR  management  process  complement  each  other. 

The  question  on  how  well  the  concept  of  operations  (CONOPs)  and  tactics,  techniques,  and  procedures 
(TTP)  described  how  the  EBP&A  process  and  the  JISR  (joint  intelligence,  surveillance,  and 
reconnaissance)  management  process  complemented  each  other  received  a  score  of  2.7.  This  indicates 
that  the  relationship  between  the  EBP  and  JISR  processes  were  not  well  defined.  One  observation  stated 
that  the  initial  TTP  changed  dramatically,  based  on  what  was  being  discovered  during  event  execution. 
That  particular  participant  saw  this  as  a  positive  thing.  Different  areas  in  the  process  received  different 
levels  of  scrutiny  during  the  event.  Additionally,  there  were  areas  of  activity  that  were  identified  that 
needed  to  be  added  to  the  TTP.  The  single  clear  observation  is  that  synchronizing  the  EBP&A  process 
with  the  JISR  concept  is  a  must. 

6.16  Component  participation  is  required  for  effects-based  planning. 

The  question  that  asked  whether  component  participation  is  required  for  EBP  received  a  score  of  4.6. 
As  the  high  score  for  this  question  indicates,  there  was  literally  no  disagreement  with  regard  to  the 
importance  of  including  the  components  collaboratively  into  the  EBP  process. 

6.17  Component  participation  in  the  JTF  mission  analysis  process  facilitated  component-level 
and  cross-component  planning  efforts. 

The  question  of  whether  component  participation  in  the  JTF  mission  analysis  process  facilitated 
component-level  and  cross-component  planning  efforts  received  a  mean  rating  of  4.1,  indicating  nearly 
everyone  agreed  that  component  participation  is  critical  to  the  mission  analysis  portion  of  the  EBP 
process.  Some  of  the  comments  indicated  that  the  horizontal  collaboration  piece  was  limited  during  this 
LOE.  It  was  also  pointed  out  that  greater  familiarity  with  the  tools  included  in  the  collaborative 
information  environment  (CIE)  would  improve  the  cross-component  planning  efforts.  One  observation 
suggested  that  as  the  “new”  functional  components  mature,  there  will  be  even  more  issues  to  coordinate 
and  develop  between  the  components  and  the  JTF  HQ.  There  may  be  a  downside  to  this  collaboration  in 
terms  of  an  "overhead  cost,"  as  far  as  the  additional  time  required  to  reach  a  consensus.  It  was  suggested 
that  component  participation  in  mission  analysis  takes  time,  which  may  be  better  spent  reading  the 
JTFC’s  assessments  and  collaborating  horizontally  to  deconflict  and  prioritize. 

6.18  Component  participation  in  the  JTF  course  of  action  development  process  facilitated 
component-level  and  cross-component  planning  efforts. 

The  question  of  whether  component  participation  in  the  JTF  COA  development  process  facilitated 
component-level  and  cross-component  planning  efforts  received  a  score  of  4.2.  Despite  the  recognition 
that  component  participation  is  a  critical  requirement  in  the  EBP  process,  a  common  theme  of  the 
written  comments  was  that  the  process  must  be  accomplished  in  less  time  than  it  took  during  the  LOE. 
One  participant  thought  a  more  refined  collaborative  process  and  better  facilitated  sessions  would  aid  in 
speeding  up  this  process. 
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7.  Conclusions 

The  concepts  entailed  in  the  EBP&A  process  were  refined  considerably  during  this  LOE.  This  was  the 
first  exposure  to  the  EBP&A  process  for  many  of  the  participants  and  the  associated  concepts  and 
processes  presented  a  novel  and  complex  planning  environment.  Participants  saw  a  huge  benefit  in 
conducting  planning  collaboratively  because  this  allows  the  expertise  of  people  representing  a  wide 
range  of  knowledge  to  be  incorporated.  Additional  positive  responses  to  the  new  EBP&A  process  were 
that  it  results  in  "out-of-the-box"  thinking  because  all  the  players  are  concentrating  on  the  effect  desired 
as  opposed  to  obtaining  a  specific  objective.  Another  player  observed  that  it  is  a  more  thoughtful 
approach,  which  should  be  done,  for  the  most  part,  before  the  crisis  begins. 

Several  consistent  themes  emerged  across  the  participants'  comments  to  the  survey  items.  The  first 
theme  is  that  the  assessment  process  will  be  critical  to  the  success  of  EBP&A  and  this  part  presents  a 
challenge.  The  second  theme  is  the  far  greater  complexity  entailed  in  the  planning  process,  and  in 
coordinating  all  the  elements  of  the  overall  planning  process  (e.g.,  using  the  ONA  to  develop  the  ETO, 
etc.).  The  third  theme  was  the  need  for  clarification  regarding  how  the  new  processes  and  procedures 
dovetail  with  existing  procedures.  The  forth  theme  was  the  need  for  articulation  of  the  relationship 
between  the  various  players  and  the  responsibilities  involved  in  planning  and  conducting  EBP&A. 
Einally,  players  noted  that  defining  success  (i.e.,  developing  MOEs  and  MOPs)  is  the  most  challenging 
aspect  of  the  EBO  process,  stating  that  it  combines  the  art  and  science  of  war  at  the  operational  level. 

The  objective  for  this  EOE  was  to  experiment  with  the  various  issues  involved  in  planning  and  conduct¬ 
ing  an  effects-based  operation  to  determine  what  worked  well  and  what  aspects  need  further  develop¬ 
ment.  Participants  thought  that  as  the  processes  mature  and  all  operators  and  planners  become  familiar 
with  the  ONA,  and  the  other  products,  the  processes  will  be  able  to  be  accomplished  in  less  time. 
Despite  the  inherent  challenges  associated  with  EBP&A,  participants  felt  that  the  benefits  will  outweigh 
the  additional  work  entailed;  the  potential  payoffs  in  terms  of  precision  and  efficiency  are  enormous. 
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